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Abstract 

A critical challenge for computer-based writing instruction is providing appropriate 
and adaptive practice. The current study examined three modes of computer-based 
writing practice with the goal of identifying those with the greatest learning and 
motivational value. High school students learned about writing strategies by studying 
lessons within the Writing Pal tutoring system and then practiced relevant strategies 
via essay-based practice, strategy practice, or game-based strategy practice. Students 
acquired strategy knowledge regarding their assigned topics, but there were no main 
effects of practice format. Similar findings were observed for students’ beliefs about 
the value of writing practice. However, the effects of practice format depended on 
prior literacy ability in subtle ways. Essay-based practice appeared to be more effect- 
ive for skilled readers, whereas less skilled readers benefitted more from game-based 
practice. Overall, multiple forms of practice opportunities may optimize benefits, 
although non-game forms of strategy practice are less preferable to students than 
game-based formats. 
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Writing instruction can be approached through diverse methods, including strat- 
egy instruction, scaffolding for prewriting and goal-setting, and peer support 
(Graham, MacArthur, & Fitzgerald, 2013; Graham & Perin, 2007). 
Underlying all of these pedagogical practices is the fundamental role of practi- 
ce—developing writers require time and multiple opportunities to hone their 
declarative and procedural knowledge of writing (Kellogg & Whiteford, 2009). 
Complex processes such as writing can also be decomposed into constituent 
tasks (see Hayes, 2012), which might then be productively practiced separately 
before (re)integrating them within the complete process (i.e., part-task vs. whole- 
task training; Wickens, Hutchins, Carolan, & Cumming, 2012, 2013). A long- 
standing plea has been for instructors to increase the number and depth of 
writing assignments that incorporate authentic writing tasks, processes, and 
practice (Applebee & Langer, 2009). 

In response, educators and researchers are increasingly using computer-based 
tools to support writing instruction, such as automated writing evaluation 
(AWE) systems that provide automated scoring and feedback on student writing 
(e.g., Shermis & Burstein, 2013; Wilson & Czik, 2016) and intelligent tutoring 
systems (ITSs) that both provide feedback and teach writing strategies (e.g., 
Calvo, O’Rourke, Jones, Yacef, & Reimann, 2011; Roscoe & McNamara, 
2013). However, enabling authentic writing practice presents a critical challenge 
for educational technology developers. Classroom teachers, although limited by 
time and resources, can nonetheless adapt to the needs of their students (e.g., 
Kiuhara, Graham, & Hawken, 2009). For example, students are often asked to 
compose prompt-based, argument essays. On reviewing students’ writing, tea- 
chers might realize that students struggle to use transitional phrases (e.g., ‘‘on 
the other hand’’) as cohesive cues. Adaptive teachers can then create supple- 
mental opportunities for practicing cohesion-building strategies before asking 
students to revise their work. Moreover, teachers could implement multiple 
forms of practice geared toward individual student needs or preferences (e.g., 
a mix of examples, games, or extra lectures). 

This degree of flexible practice can be difficult to implement in educational 
technologies because personalization and adaptivity must be “‘built in” to the 
system. Developers must decide how and whether to expend limited resources to 
craft diverse instructional elements, along with the underlying algorithms that 
coordinate these resources. There is currently little empirically supported 
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guidance about which modes of practice are more or less effective across varying 
circumstances. Should computer-based tools focus on a single “‘best”’ practice 
approach or strive to provide multiple forms of practice? Should any types of 
practice be discontinued, and if so, when? For whom? 

One relevant issue is student engagement. Practice is not necessarily fun and 
requires significant, sustained effort (Kellogg & Whiteford, 2009). Writing, in 
particular, evokes student attitudes ranging from enjoyment to extreme anxiety 
(e.g., Troia, Shankland, & Wolbers, 2012), and developers of educational tech- 
nologies for writing must take students’ motivation into account. Additionally, 
personalized writing strategy instruction and practice may depend on learners’ 
prior literacy ability. Reading and writing—as activities that depend on similar 
underlying lexical, syntactic, semantic, and rhetorical knowledge and skills— 
share a close connection (Fitzgerald & Shanahan, 2000; Shanahan, 2015; 
Stotsky, 1983). Students’ ability (or lack thereof) to access existing literacy 
skills and knowledge potentially influences the benefits of various forms of 
writing practice. 

To address these issues, this article contrasts three forms of practice that 
could be implemented in computer-supported writing strategy instruction: 
essay-based practice, strategy practice, and game-based strategy practice. Each 
format possesses different strengths and weaknesses related to demands on prior 
literacy abilities or potential for engagement. This work is conducted within a 
population of high school adolescent writers, who represent a crucial target for 
writing instruction. 


Computer-Supported Writing Practice 


Writing is a complex activity that entails nonlinear and iterative processes of 
idea generation, organization, translation, and refinement (e.g., Hayes, 2012), 
and learning how to coordinate these activities is critical for writing develop- 
ment. To support students’ cognitive and motivational navigation of these 
tasks—whether through “‘offline’’ or computer-based support—educators must 
provide instruction that emphasizes concrete writing strategies and provides the 
background knowledge needed to employ the strategies (Graham, Harris, &, 
Chambers, 2016; Graham & Perin, 2007). Students must then develop their 
understanding and proficiency with these strategies via iterative practice oppor- 
tunities (Kellogg & Whiteford, 2009) in some form (e.g., essay-based practice, 
strategy practice, and game-based practice). 


Essay-based practice. Essay-based practice—writing and _ revising complete 
essays—is perhaps the canonical and most traditional format. Within an AWE 
or ITS environment, students might be assigned a writing prompt and given a 
virtual ‘“‘scratch pad” to take notes or make an outline. Subsequently, they can put 
these plans into action by authoring essays, which are then submitted for 
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automated scoring and formative feedback to guide revisions. Across multiple 
cycles, mindful students can reflect on whether their essays (or specific sections) 
are improving, whether newly acquired strategies are working, and whether the 
overall writing task seems more enjoyable. In other words, learners have the 
opportunity to integrate and coordinate multiple strategies and skills, and this 
type of authentic “whole task” practice can support strategy acquisition (Lim, 
Reiser, & Olin, 2009; van Merriénboer & Kirschner, 2017). A number of commer- 
cially available AWE systems enable such practice, allowing students to write, 
receive feedback, and revise: Educational Testing Service’s e-rater/Criterion 
(Burstein, Tetrault, & Madnani, 2013), Pearson’s WriteToLearn (Foltz, Streeter, 
Lochbaum, & Landauer, 2013), and Measurement Incorporated’s Project Essay 
Grade (Wilson, Olinghouse, & Andrada, 2014). 

The richness of essay-based practice also presents cognitive and motivational 
challenges. As with any skill that entails multiple stages or components, practi- 
cing the whole task can be overwhelming to novice learners (see Salden, Paas, & 
van Merriénboer, 2006; Wickens et al., 2013). In learning to write, students often 
struggle to coordinate the multiple, requisite cognitive processes of essay com- 
position (e.g., planning, drafting, and revising) simultaneously (Braaksma, 
Riylaarsdam, van den Bergh, & van Hout-Wolters, 2004; Olive, 2014). 
As such, whole-task writing practice could be particularly challenging for stu- 
dents with lower reading and writing skills. Indeed, students’ reading skills 
correlate strongly with writing skills and have been found to meaningfully influ- 
ence the effects of writing strategy acquisition (Allen, Snow, Crossley, Jackson, 
& McNamara, 2014; Roscoe, Jacovina, Harry, Russell, & McNamara, 2015; 
Shanahan, 2015). Skilled readers have stronger vocabulary knowledge and lan- 
guage fluency to draw upon, and they already possess a better understanding of 
the features of good text. These writers might benefit from whole-task, essay- 
based practice that encourages (or requires) integrating prior knowledge and 
newly learned strategies in a complete composition. Less skilled readers, by 
contrast, may struggle with internalizing new strategies and encounter difficulties 
when attempting to enact these strategies while planning, drafting, and revising 
an entire essay. 


Strategy practice. An alternative approach, targeted strategy practice, decomposes 
the writing process into tractable components that can be practiced and mas- 
tered individually before tackling a complete essay. Instead of practicing many 
strategies simultaneously, students benefit from honing in on just one or two 
explicit and actionable strategies. Such practice likely helps learners of all read- 
ing ability levels, with particular benefits for less skilled readers (Gillespie & 
Graham, 2014) by enabling practice of key concepts and procedures with 
fewer initial cognitive demands and distractions. Research on part-task training 
of complex cognitive tasks suggests that this approach can lead to more efficient 
and robust skill acquisition (Wickens et al., 2012, 2013). 
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Research has indeed strongly supported the value of strategy instruction for 
students’ writing development (Graham & Perin, 2007; Graham et al., 2016; 
Newell et al., 2011). Studies have demonstrated that teaching students about 
concrete writing strategies (e.g., using precise language, elaborating, citing ver- 
ifiable rather than hypothetical evidence, and considering counterarguments), 
providing them with the underlying background and genre knowledge, and 
enabling practice opportunities, support writers of diverse ages and ability 
levels. Benefits of strategy instruction have been documented for learners in 
college (e.g., Butler & Britt, 2011; MacArthur, Philippakos, & Ianetta, 2015), 
middle and high school (e.g., De La Paz & Felton, 2010; Graham & Perin, 2007), 
and elementary school (e.g., De La Paz & Sherman, 2013; Ferretti, Lewis, & 
Andrews-Weckerly, 2009) and seem particularly beneficial for struggling writers 
with weaker reading skills or learning disabilities. 


Game-based practice. Educational games and game-based practice offer another 
route for providing computer-based writing strategy practice (Habgood & 
Ainsworth, 2011; Jackson & McNamara, 2013; Proske, Roscoe, & 
McNamara, 2014). Although far from a motivational panacea (see Wouters, 
van Nimwegen, van Oostendorp, & van der Spek, 2014), educational games 
can leverage learners’ intrinsic enjoyment of play to inspire improved interest 
in associated learning tasks (Ryan, Rigby, & Przybylski, 2006). In terms of 
strategy-based practice, typical practice activities (e.g., analyzing examples of 
text using an assigned strategy) can be embedded within game features and 
narratives to potentially make them more engaging. Moreover, as students gra- 
dually add to their repertoire of writing skills, they feel more empowered or 
motivated to write (Troia et al., 2012). Thus, game-based strategy practice 
may synthesize the benefits of both part-task training and increased motiva- 
tional engagement. 

Researchers and educators have explored the use of games in writing instruc- 
tion in a variety of ways (Colby, 2017), including games that link writing to 
game mechanics (i.e., players must write well to succeed in the game; e.g., Barab, 
Pettyjohn, Gresalfi, Volk, & Solomou, 2012; Dickey, 2011; Liao, Chang, & 
Chan, 2018). For example, Dickey (2011) observed that a narrative game envir- 
onment for argument writing, Murder on Grimm Isle, was effective in engaging 
college students’ interest and curiosity, and students were able to use lessons 
learned from the game to improve prewriting strategies (e.g., evidence genera- 
tion). Similarly, Barab et al. (2012) studied middle school students’ argumenta- 
tive writing and engagement with the Plague: Modern Prometheus segment of 
Quest Atlantis. In the game, the alignment of students’ written thesis statements, 
evidence, and their audience influenced their interactions with game characters 
(i.e., arguments had narrative consequences). In comparison to a non-game 
comparison lesson, students who wrote and practiced in the game environment 
exhibited higher learning gains (i.e., improved persuasive writing quality and 
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genre knowledge) and motivation. Finally, Warren and colleagues (Warren, 
Dondlinger, & Barab, 2009; Warren, Dondlinger, Stein, & Barab, 2008) have 
investigated benefits of the Anytown multiuser virtual environment for support- 
ing elementary school students’ writing. In this game-like environment, students 
take on the role of reporters to write about a series of mysterious events. 
Researchers found that the game-like curriculum encouraged more voluntary 
writing practice and increased writing achievement in comparison to a tradi- 
tional, non-game comparison curriculum. 


Contrasting formats with Writing Pal. Although prior research has provided empiri- 
cal evidence for the plausibility and effectiveness of various practice formats, the 
relative merits of these formats have rarely been tested directly (see Proske et al., 
2014). Another resource for research on writing strategy acquisition and practi- 
ce—the Writing Pal (W-Pal) ITS—has a modular design that affords such com- 
parisons. In W-Pal, high school adolescents are taught writing strategies via a 
series of animated, 5-minute multimedia lessons. The strategies developed for 
W-Pal modules were informed by English education experts and prior literature 
on argumentative writing and revising strategies (e.g., Butler & Britt, 2011; 
De La Paz & Graham, 2002; Graham & Perin, 2007; Roscoe, Allen, Weston, 
Crossley, & McNamara, 2014). Students have the opportunity to practice those 
strategies by playing games or writing practice essays with automated formative 
feedback (Allen, Crossley, & Snow, 2014; Roscoe, Brandon, Snow, & 
McNamara, 2013; and see Method section). 

The modular and multipronged design of W-Pal makes it ideal for studying 
the role of distinct components of writing instruction in combination or isola- 
tion. For instance, in a module on freewriting (see Elbow, 1989; Li, 2007), 
students are taught a FAST mnemonic for generating ideas (i.e., Figure out 
the prompt, Ask and answer questions, Support ideas with evidence, and 
Think about the other side). After studying lessons on each step of the mnemo- 
nic, students can play Freewrite Flash to practice. In that game, students free- 
write on a given prompt for about 2 minutes and earn points by generating a 
diverse pool of ideas. The game rewards students for typing quickly and con- 
tinuously, and for including a large variety of key words and concepts. In the 
game, students are not asked to organize their freewrite or translate it into an 
essay; idea generation strategies are practiced outside of the context of other 
writing tasks and demands. Alternatively, or additionally, students can practice 
freewriting in the context of composing essays. The essay authoring interface 
includes a “scratch pad” for students to freewrite (or write down other notes and 
outlines). Thus, multiple forms of practice are available. 

W-Pal researchers have contrasted the benefits of using the full W-Pal system 
(i.e., lessons, games, and essay practice) versus intensive essay-based practice 
(i.e., using only the essay writing and feedback tools of W-Pal) over multiple 
sessions. Both groups made modest gains in writing quality and writing strategy 
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knowledge. However, students who had access to game-based practice expressed 
more positive writing self-efficacy and motivation and were able to articulate 
more new strategy concepts at posttest (Allen et al., 2014; Roscoe et al., 2013). 
For students who played the mini-games, game scores predicted strategy knowl- 
edge acquisition and game enjoyment was associated with whether students 
perceived the games as helpful for learning. Thus, the cognitive and motivational 
benefits of game-based writing practice are linked. 

Proske et al. (2014) specifically examined motivational and learning outcomes 
for four different types of practice: game-based, question-based, model-based, 
and writing-based. German undergraduate students enrolled in English language 
classes first learned strategies for writing the introduction section of an essay via 
W-Pal lesson videos. Students then practiced these strategies by playing a 
science-fiction themed game (1.e., rescuing spaceships) in which they identified 
thesis statements and stylistic techniques in example paragraphs (i.e., game- 
based practice condition). For question-based practice, students completed the 
same strategy identification tasks but all game elements were removed—the task 
consisted of a series of example paragraphs and probes about which strategies 
were used. In model-based practice, students studied examples of complete intro- 
duction paragraphs (i.e., included a thesis statement, preview of arguments, and 
an attention-grabbing hook) and were prompted to attend to and analyze these 
elements. Finally, in writing-based practice, students were instructed to write 
introductory paragraphs in response to three different prompts. 

Students made equivalent gains in strategy knowledge across all four practice 
formats (i.e., no effect of condition). On measures of students’ ability to strate- 
gically revise prewritten paragraphs, writing-based practice was somewhat less 
effective than other conditions, whereas model-based practice was slightly more 
effective. Model-based practice may have been beneficial because it allowed 
students to hone in on and analyze important features of introductions without 
the additional cognitive burden of generating text or navigating distracting game 
mechanics. In terms of motivation, game-based practice was perceived as more 
interesting and engaging than question-based practice. Thus, for tasks that were 
highly isomorphic, game-based formats were more motivating. Interestingly, 
game-based practice did not differ significantly from model-based or writing- 
based practice in terms of motivation. Studying complete examples or authoring 
examples might have been perceived as more authentic, which translated into 
similar levels of engagement as an amusing game-based format. 

The study conducted by Proske et al. (2014) revealed positive aspects of 
model-based practice and uniquely contrasted multiple writing practice formats 
simultaneously, but it had a few potential limitations. First, participants were 
nonnative English speakers using learning materials written in English. 
The challenge or novelty of using W-Pal in that context likely influenced learn- 
ing and motivational outcomes. In addition, W-Pal was ostensibly developed for 
high school-aged adolescents, and participants in the study were university 
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students. It is unclear whether observed results would generalize to younger 
learners. Finally, the study assessed learning for only one content area and 
did not examine students’ prior reading ability. 

In sum, practicing relatively novel writing strategies can occur in a variety of 
formats, including traditional essay-based methods or strategy-based approaches 
that focus upon key strategies in isolation. The latter form of part-task practice 
could facilitate learning and motivation, particularly for less skilled learners, by 
helping to break down and emphasize fewer strategies and procedures at any one 
time. In addition, strategy practice can be embedded within games by adding 
game features (e.g., points, levels, and competition) and narratives (e.g., piloting 
a spaceship) to encourage more positive engagement. 


Research Questions 


The current study is guided by three research questions informed by prior 
research on writing instruction, practice, and related individual differences. 
Importantly, this work continues the uncommon practice of contrasting multiple 
formats rather than focusing only a single format. First, we consider whether the 
format of writing strategy practice in a computer-based system influences strat- 
egy knowledge acquisition (RQ1). Second, we consider whether practice format 
influences changes in attitudes toward writing and practice (RQ2). Finally, we 
assess whether the effects of practice format are differentially beneficial for more 
and less skilled readers (RQ3). We hypothesize that strategy practice and game- 
based strategy practice will be more beneficial to novice writers due to focusing 
practice efforts on specific strategies and that game-based practice will be the 
most motivating. Strategy-based and game-based practice effects may be stron- 
ger for less skilled readers, due to lower demands on their literacy skills. 


Method 
W-Pal and Learning Topics 


The current study was conducted using learning materials, activities, and tools 
obtained from the W-Pal tutoring system. In this experiment, students studied 
one of two writing strategy modules: Body Building or Conclusion Building. 


Body Building. In the Body Building module, students learn strategies for author- 
ing the paragraphs that form the middle or “tbody” of an essay. In a series of 
four lessons (i.e., Overview, Topic Sentences, Evidence Sentences, and 
Strengthening Your Evidence), students are taught to organize body paragraphs 
around concise arguments that are supported with factual and verifiable evi- 
dence. Students also learn how to recognize problematic evidence that lacks 
detail or relies too heavily on hypotheticals or opinions. These strategies are 
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Prompt: Does life imitate art, or does art imitate life? 


Life can imitate art, as people often use optimistic art forms, such as 
pop music, to survive difficult situations; this can help them to change 
their disposition and negative feelings about their lives. Upbeat music, 
such as techno or pop music, typically portrays the world as a happy 


and lively place. When people listen to these genres of music, they are 
typically cheered up and begin viewing the world in a more optimistic 
way. In many cases, people choose to listen to happy music, because 


Bm The topic sentence of this body paragraph is too long. 
This body paragraph does not contain adequate evidence. 


This body paragraph correctly uses the CASE strategy. 


Figure |. Fix It: Bodies game from the Writing Pal Body Building module. 


summarized by the C.A.S.E. mnemonic: Concise Argument and Supporting 
Evidence. 

To practice relevant strategies, students can play two games: Fix It: Bodies 
and RoBoCo. In Fix It: Bodies (see Figure 1), students are presented with four 
example body paragraphs exhibiting errors in one or more critical elements (e.g., 
missing topic sentence or inappropriate evidence). Students earn points for 
identifying the problem and recommending solutions (e.g., replacing off-topic 
evidence with relevant evidence). Correct responses earn points along with 
“golden circuits.” In the final stage, students play a Sudoku-like puzzle where 
the golden circuits can be spent to obtain solution hints. Thus, better perfor- 
mance during the “learning” portion of the game translates into helpful 
resources in the “playing” portion. Game duration naturally varies across stu- 
dents, but a single iteration of this game requires about 5 minutes. 

In RoBoCo (see Figure 2), students take on the role of a robot builder in the 
Robot Body Company. Students earn robot parts by authoring two body para- 
graphs (i.e., topic sentence and supporting evidence) in response to a prompt. 
Throughout this process, students can request strategy hints about good versus 
poor body paragraphs. For example, hints encourage students to provide suffi- 
cient elaboration, avoid overly subjective language when presenting evidence 
(e.g., “I believe’? and “I feel’), avoid purely hypothetical examples (e.g., “‘if- 
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Prompt: People today have many choices. For instance, thirty years ago most 
television viewers could choose from only a few channels; today there are 
hundreds of channels available. Choices are not just limited to the media. People 
have more options in almost every area of life including clothing, universities, 
cars, and music. Having so many choices may lead people to doubt the selections 
they make. 


i Does having a large number of options to choose from increase or decrease 
satisfaction with the choices people make? 


/ = 


; When people have too many choices, they can be overcome with indecision 


Every choice we make requires a bit of cognitive or mental energy. The more 
choices we are faced with, the more attributes and options we have to analyze. 
As a result, an abundance of choices can create cognitive overload. The human 
mind simply cannot handle all of the many variables and potential outcomes. In 
this situation, the decision-maker might shut down. They become overwhelmed 
and cannot make any decision at all. 


Figure 2. RoBoCo game from the Writing Pal Body Building module. 


then’’), and employ causal language when making claims (e.g., “because” and 
“leads to’’). Natural language processing (NLP) metrics (see McNamara, 
Crossley, & Roscoe, 2013) assess student contributions based on these linguistic 
properties, and higher scoring topic and evidence sentences earn more parts to 
be assembled into robot models. At the end of the game, students’ robots are 
displayed at the “Annual Robot Show” and the game score is expressed as 
company profits. Thus, generating better paragraphs results in more engaging 
visual displays (more robots) and score rewards. A single iteration of the game 
requires about 10 minutes depending on the student. 


Conclusion Building. In the Conclusion Building module, students learn strategies 
for authoring the final summary of an essay. In a series of four lessons (i.e., 
Overview, Summarize the Essay, Close the Essay, and Hold the Reader’s 
Interest), students learn to remind the reader of main ideas and to bring the 
essay to a clear, engaging end. For example, students are taught to avoid 
introducing new evidence and to link themes from the essay to broader 
issues or calls to action. These strategies are summarized via the R.E.C.A.P. 
mnemonic: Restate the thesis, Explain how thesis was supported, Close 
the essay, Avoid new arguments and evidence, and Present ideas in an inter- 
esting way. 
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from these losses. 


Figure 3. LockDown game from the Writing Pal Conclusion Building module. 


Students can practice these strategies by playing several games, including Fix 
It: Conclusions and LockDown. Fix It: Conclusions has identical design, dura- 
tion, and game mechanics as the Body Building version of the game but uses 
example conclusion paragraphs. In LockDown (see Figure 3), students take on 
the role of an agent working for the Writing Intelligence Agency. To prevent 
computer hackers from attacking the system, students must ‘“‘lock down”’ a series 
of three essays by generating conclusion paragraphs. Students receive an essay 
prompt and an outline of main arguments and evidence, and have a maximum of 
5 minutes to author a relevant conclusion paragraph. Throughout the task, 
students can access reminders regarding the R.E.C.A.P. mnemonic and strate- 
gies. Simple, underlying NLP-based metrics (e.g., keyword comparisons) assess 
linguistic indicators that students have restated the thesis and summarized the 
main ideas, while avoiding repeating the specific points of evidence or adding 
new evidence. 

After each round, students receive feedback on potential weaknesses in their 
paragraphs, such as reminders to “Summarize the thesis and arguments, but do 
not discuss specific evidence or examples.” Finally, at the end of the game, 
students receive an overall score, and are either successful or unsuccessful in 
obstructing the hackers. A single iteration of the game requires about 10 minutes 
depending on the student. 
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Participants 


Participants were 163 high school students (aged 15 to 17 years), recruited from 
local schools in the southwestern United States, who completed all components of 
the study. High school adolescents represent a critical target for innovative writ- 
ing instruction for many reasons. For example, national samples of over 24,000 
Grade 8 and 28,000 Grade 12 students were examined in 2012 (National Center 
for Education Statistics, 2012). At both grade levels, only one quarter of adoles- 
cents wrote at a proficient (24%) or advanced (3%) level. About half of all 
adolescents wrote at just a basic level (52% to 54%) and one fifth wrote at a 
below basic level (20% to 21%). These worrying patterns mirror NAEP reports 
from 2007 (Salahu-Din, Persky, & Miller, 2008) and 2002 (Persky, Daane, & Jin, 
2003)—over time, adolescents’ writing deficits have remained steady rather than 
improving. In addition, surveys of high school teachers suggest that they are 
underprepared to enact best practices for writing instruction, and few assignments 
involve substantive research, analysis, or argument (Kiuhara et al., 2009). This is a 
concern because many high school students are preparing for their professional 
and academic futures, and writing challenges are a significant hurdle that impacts 
students’ college and job prospects. 

Students were recruited via flyers distributed by their classroom teachers and 
received US$30.00 for their participation. Most students completed the study in 
an after-school setting in their own schools, although researchers accommodated 
several students’ schedules by offering evening and weekend sessions in a uni- 
versity lab setting. Students were separated by one or more desk spaces, or with 
tall dividers between them, to prevent viewing of others’ work or computer 
screens. Students were also monitored by researchers at all times, and research- 
ers enforced a quiet environment of individual work. 

Demographic data were obtained via a background survey completed at the 
beginning of the study. A slight majority of students self-identified as female 
(57.7%), and students were identified as primarily Hispanic (42.3%) or 
Caucasian (29.4%), although the sample also included African American 
(14.1%), Asian (2.5%), or other backgrounds (11.7%). Most students reported 
English as their first language (80.4%). Finally, slightly less than half of students 
self-reported a GPA greater than 3.0 (42.9%). There were no differences in 
background across students who studied either topic and no differences across 
the three experimental conditions. 

Students were asked to estimate their daily computer use and overall discom- 
fort with using computers. Students expressed a range of responses: little to no 
use (6.7%), less than | hour (22.1%), 1 to 2 hours (30.7%), 3 to 4 hours (25.2%), 
and 5 or more hours (15.3%), but most students used computers at least 1 hour 
per day (71.3%). Computer discomfort was rated on a 6-point scale from “1” 
(very comfortable) to “6” (very uncomfortable). Students reported a high level 
of comfort (M@= 1.7, SD =0.8). Computer use and comfort did not differ across 
learning domain or experimental condition. 
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Procedure and Conditions 


The study occurred over three phases in a single 2-hour session per student. In 
the pretest phase, participants completed an assessment of their prior writing 
strategy knowledge pertaining to body and conclusion paragraphs. Students also 
completed a survey measure regarding their attitudes toward writing strategies 
and practice. Finally, as a measure of prior reading ability, students completed 
the Gates-MacGinitie Vocabulary Test. 

In the learning and practice phase, all students interacted with strategy lessons 
and practice activities for either Body Building or Conclusion Building topics. 
Students were randomly assigned to one of three practice activities within their 
topic: essay practice, strategy practice, or game-based strategy practice. 

Finally, in the posttest phase, students completed a second assessment of their 
strategy knowledge. Because each student only studied one of the two topics, 
questions on the “‘studied’’ topic assessed learning while questions on the 
“unstudied” topic served as a control. Students also completed a posttest 
survey of writing attitudes. 


Essay practice condition. Essay-based practice was implemented by assigning stu- 
dents to interact with W-Pal AWE tools. Students were given 25 minutes to 
compose a prompt-based argument essay, after which the essays were automa- 
tically submitted to W-Pal for scoring and formative feedback. Students were 
then allotted 10 additional minutes to revise. Overall, students spent about 40 
minutes in this writing practice activity. 

Students were randomly assigned to one of two practice prompts (see 
Appendix A). They were instructed to take a stance on the assigned issue and 
then to develop and defend their point of view on the topic using relevant 
evidence and examples. Two prompts were chosen to ensure that any observed 
results were not simply due to a single idiosyncratic prompt selection. Analyses 
(not reported for brevity) confirmed that both prompts performed similarly. 
There were no differences at pretest, posttest, or in strategy gains. 


Game-based strategy practice condition. Students who interacted with the Body 
Building module played two games: RoBoCo and Fix It: Body Building. 
Students who interacted with the Conclusion Building module also played two 
games: Lockdown and Fix It: Conclusion Building. Both games were played for 
about 20 minutes. Thus, game-based practice duration closely matched the 
duration of essay-based practice. 


Strategy practice condition. We created “‘non-game”’ versions of all four practice 
games. Specifically, RoBoCo was replaced with the Topic and Evidence Writer 
practice task; Fix It: Body Building was replaced with the Body Paragraph 
Editor task; Lockdown was replaced with the Conclusion Writer task; and Fix 
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It: Conclusion Building was replaced with the Conclusion Editor task. All game 
narratives, mechanics, character graphics, and similar elements were removed, 
but the writing and strategy identification tasks remained unchanged. Students 
continued to receive feedback and other indicators of right and wrong answers. 
For example, in the Body Paragraph Editor task (i.e., the non-game alternative to 
Fix It: Body Building), students read a series of paragraphs, identified the primary 
flaw exhibited, and attempted to recommend a relevant solution. However, the 
fictional narrative of collecting “golden circuits” was removed, and students did 
not complete a Sudoku-like puzzle or earn a “‘game score.” 

Students practiced their two assigned activities for 20 minutes each, thus 
equating time-on-task across all three conditions. 


Measures 


Strategy knowledge assessments. Students’ knowledge of Body Building and 
Conclusion Building were assessed via two quizzes (see Appendix B). Each 
quiz comprised eight multiple-choice questions related to writing strategies 
taught in W-Pal modules. For example, students were asked about the functions 
of body paragraphs or conclusion paragraphs, to identify strategies for improv- 
ing each type of paragraph, and to recognize features of “good” and ‘‘bad” 
example paragraphs. Incorrect foils were crafted to seem plausible and to 
align with typical misconceptions. The order of questions and answer foils 
was randomized across pretest and posttest versions to ensure that questions 
and answers never appeared in the same position. 

An example Body Building question asked, ““Why is it important to include a 
body paragraph in an essay?” The correct answer stated that “Body paragraphs 
support the arguments in the essay.” Incorrect foils suggested alternative func- 
tions including making an essay longer, stating the purpose of the essay, and 
providing a summary of the arguments. An example Conclusion Building ques- 
tion asked, “Which of the following is a good way to summarize your main ideas 
in the conclusion paragraph?” The correct choice stated, ‘““Describe how your 
ideas connect to the main argument.” Incorrect foils suggested strategies such as 
simply making a bulleted list, adding in new dates and facts, or ignoring the issue 
because main ideas were already summarized in the body of the essay. 

Knowledge assessments were scored by tallying the number of correct 
responses. Thus, each topic quiz could range in score from “0” to “8.” 


Writing attitudes survey. Students’ attitudes toward writing, strategies, and practice 
were examined via 12 probes constructed by the researchers (see Appendix C). 
Students rated their agreement with a series of straightforward statements about 
writing and practice using a 6-point Likert scale. 

Because this measure was not based on a specific, preexisting scale, an 
exploratory factor analysis using principal axis factoring and oblique rotation 
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examined whether all pretest attitude items loaded on a single factor or were 
perhaps separable. In fact, three factors emerged with eigenvalues greater than 
1.00 and which cumulatively accounted for 74.82% of the variance. Six items 
loaded on the first factor (loadings > .62), which comprised attitudes regarding 
enjoyment and interest in writing practice and strategies (e.g., ““Writing practice is 
entertaining” and “‘I am excited about writing strategies’). Two items loaded on 
the second factor (loadings > .72), which related to perceptions of importance 
and value of strategies and practice (i.e., “It is important for me to know writing 
strategies” and “I learn valuable things through writing practice’). Finally, three 
items loaded on a third item (loadings > .81) and appeared to relate to general 
attitudes about writing (e.g., “I think writing is interesting” and ‘“‘Writing fasci- 
nates me’’). One item failed to load on any factor (“During writing practice 
I need something to grab my attention”) and was excluded. Similar analyses 
were conducted for posttest attitude items—three factors again emerged that 
comprised the exact same items. 

Based on the aforementioned patterns, three composite pretest and posttest 
attitude scores were created by averaging the ratings for a given factor: interest in 
strategies and practice, value of strategies and practice, and interest in writing. 


Prior reading ability. Prior reading ability and vocabulary knowledge were assessed 
via the Gates-MacGinitie Vocabulary Test (4th ed., Form S) level 10/12 
(MacGinitie & MacGinitie, 1989). The vocabulary section contains 45 simple 
sentences with each sentence containing an underlined word. Students were 
asked to select the word that was most closely related to the underlined word 
from a list of five choices. Students were given the standard instructions, two 
practice questions, and 10 minutes to finish the test. Previous research has shown 
positive correlations between scores on the Gates-MacGinitie Vocabulary Test 
and reading comprehension ability (e.g., Allen et al., 2014; r=.79). 


Results 


Whereas prior research focused on a single topic (i.e., Introduction Building), 
the current study investigated two topics (i.e., Body Building or Conclusion 
Building), although each student studied only one topic. We first analyze out- 
comes for the two topics separately to determine whether there are meaningful 
topic-specific effects. If patterns are overall similar for both topics, they can then 
be collapsed in subsequent analyses of the effects of prior reading ability. 


Body Building Topic 


Prior ability and pretest equivalence. Several univariate analyses of variance 
(ANOVAs) were conducted to compare prior ability and attitudes across con- 
ditions at the outset of the study. For students who studied the Body Building 
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Table |. Mean Prior Ability, Knowledge Assessment Scores, and Writing Attitudes by 
Condition for Students Who Studied the Body Building Topic. 


Practice Format Pretest Equivalence 
Essay Game Strategy 

Measure (n= 28) (n= 27) (n= 26) F(2,78) p partial 1” 
Reading Ability 

GMVT 27.5 (8.7) 25.6 (8.5) 25.5 (7.8) <I.00 .603 013 
Pretest Knowledge 

Body 5.9 (1.3) 5.0 (1.7) 5.2 (2.0) 2.05 .136 .050 

Conclusion 4.9 (1.5) 45 (1.5) 44(1.9)  <1.00 509 017 
Pretest Attitudes 

Practice Interest 3.4(0.9) 3.4 (1.1)  3.7(1.0)  <1.00 .430 021 

Practice Value 4.9 (0.9) 5.0 (0.8) 5.1 (0.7)  <!1.00 .610 .013 

Writing Interest 3.9(1.3) 4.4 (1.1) 4.3 (1.0) 1.42 249 .035 


Overall Change from Pretest 


Posttest Knowledge 


Body 6.2 (1.6) 5.6(1.6) 54(2.0) 5.07 027 061 

Conclusion 49 (1.5) 43(1.2) 4.2 (1.6)  <!I.00 529 005 
Posttest Attitudes 

Practice Interest 3.6 (1.1) 3.7 (1.1) 3.8(1.0) 3.67 059 045 

Practice Value 5.0 (1.0) 50 (1.0) 52(08) <!1.00 537 005 

Writing Interest 3.7 (1.3) 4.1 (1.2) 3.9 (1.4) 9.82 .002 12 


Note. GMVT = Gates-MacGinitie Vocabulary Test. 


topic, there were no significant differences across conditions for prior reading 
ability, pretest knowledge of Body Building or Conclusion Building strategies, or 
pretest attitudes about writing and writing practice (see Table 1). 


Main effects of practice. Gains in strategy knowledge from pretest to posttest were 
assessed via repeated measures ANOVA with practice format entered as a 
between-subjects variable. After studying lessons on Body Building and enga- 
ging in assigned practice activities, students made moderate gains in Body 
Building strategy knowledge at the posttest, and exhibited no gains in 
Conclusion Building strategy knowledge (i.e., unstudied topic). Thus, interacting 
with instructional materials from W-Pal appeared to facilitate acquisition of 
strategy knowledge. 

Students also demonstrated changes, albeit mixed, in their attitudes toward 
writing and writing practice. Students reported a significant decrease in writing 
interest from pretest to posttest. This result perhaps underscores the effort 
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associated with practice. Learning new skills and engaging in practice can gen- 
erate fatigue, which may translate into transitory decreases in general motivation 
for writing. However, we also observed a small increase in students’ interest in 
writing practice. Thus, interacting with some of the W-Pal practice activities 
might have made the act of practicing more pleasant (despite also leading to 
fatigue). There were no changes in students’ attitudes about the value of prac- 
tice, which tended to be positive at pretest and remain so throughout the study 
(i.e., students endorsed the value of practice). 


Interaction with practice format. There were no significant interactions between 
practice format and learning gains for either the Body Building topic (i.e., stu- 
died topic), F(2,78) < 1.00, p=.555, partial n°=.015, or Conclusion Building 
topic (i.e., unstudied topic), F(2,78)<1.00, p=.839, partial n?=.004. 
Likewise, there were no significant interactions between practice format and 
changes in interest in writing practice, F(2,78)<1.00, p=.809, partial 
7 = .005, value of writing practice, F(2,78) < 1.00, p=.893, partial 1° = .003, 
or general attitudes toward writing, F(2,78) < 1.00, p=.710, partial y7= .009. 


Conclusion Building Topic 


Prior ability and pretest. For students who studied Conclusion Building, there were 
no significant differences across conditions for prior reading ability, pretest 
knowledge of Body Building or Conclusion Building strategies, or pretest atti- 
tudes about writing (see Table 2). 


Main effects of practice. Students exhibited significant gains in Conclusion 
Building strategy knowledge. Students also demonstrated a significant decrease 
in scores on the Body Building strategy knowledge quiz (i.e., the unstudied 
topic). As stated earlier, studying instructional materials provided by W-Pal 
seemed to support strategy knowledge acquisition for the studied topic. 
Students’ scores for the unstudied topic perhaps decreased because those ques- 
tions were not salient during study and practice, and thus they made no effort to 
learn those topics. Another possibility is that negative transfer (see Schwartz, 
Chase, & Bransford, 2012) occurred between topics. For example, the properties 
of good conclusions (i.e., avoiding offering new evidence) somewhat conflict 
with properties of good body paragraphs (i.e., providing elaborated evidence), 
perhaps leading to confusion. After studying Conclusion Building strategies 
(and without the opportunity to study Body Building strategies), students may 
have misapplied the new strategies. 

No significant positive or negative changes were observed for writing atti- 
tudes and interests. Except for a small gain in perceptions of the importance of 
writing practice, students’ attitudes were stable across pretest and posttest 
administrations of the survey. 
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Table 2. Mean Prior Ability, Knowledge Assessment Scores, and Writing Attitudes by 
Condition for Students Who Studied the Conclusion Building Topic. 


Practice Format Pretest Equivalence 
Essay Game Strategy 

Measure (n= 27) (n= 28) (n=27) F (2,78) p partial 1” 
Reading Ability 

GMVT 22.8 (8.4) 25.7 (8.5) 27.2 (8.8) 1.82 .168 044 
Pretest Knowledge 

Body 5.5 (1.9) 5.2 (1.7) 5.6 (2.0)  <!I.00 .69| .009 

Conclusion 45 (1.5) 47 (1.5) 46(1.5)  <1.00 893 .003 
Pretest Attitudes 

Practice Interest 3.0(1.0) 3.2 (1.0) 34(I.1)  <!I.00 447 .020 

Practice Value 4.3(1.2) 46(1.0) 48(1.2) 1.05 354 026 

Writing Interest 3.5 (1.2) 4.0 (1.1) 4.3°(1.3) 2.65 077 .063 


Overall Change from Pretest 


Posttest Knowledge 


Body 5.4 (1.8) 44(2.0) 50(24) 7.68 .007 .089 

Conclusion 5.4 (2.4) 5.4(1.9) 5.3 (2.4) 11.74 .001 129 
Posttest Attitudes 

Practice Interest 3.0(1.0) 3.5 (1.2) 3.4(1.0) < 1.00 359 ll 

Practice Value 5.0 (1.0) 50 (1.0) 48(1.0) 3.34 072 041 


Writing Interest 3.4 (1.4) 4.0(1.3) 43(12) <100 645 003 


Note. GMVT = Gates-MacGinitie Vocabulary Test. 


Effects of practice format. There were no significant interactions between practice 
format and learning gains for either the Conclusion Building topic (i.e., studied 
topic), F(2,78) < 1.00, p=.865, partial n?=.004, or Body Building topic (ie., 
unstudied topic), F(2,78) < 1.34, p=.267, partial n*?=.033. Similarly, there 
were no significant interactions between practice format and changes in interest 
in writing practice, F(2,78) = 1.92, p=.154, partial n*=.046, value of writing 
practice, F(2,78)= 1.04, p= .360, partial 1 = .026, or general attitudes toward 
writing, F(2,78) < 1.00, p=.707, partial n*?= .009. 


Effects of Prior Reading Ability 


The preceding analyses suggest that students learned about writing strategies via 
W-Pal although attitudinal outcomes were mixed. More importantly, these 
results also suggested that the main effects of practice format were largely 
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equivalent; no differences in learning or attitude were observed depending on 
whether students practiced by writing an essay, engaging in targeted strategy 
practice, or playing strategy-driven games. In our final analyses, we evaluate 
how the effects of practice format potentially depend on students’ reading skills. 
As described previously, less skilled readers could find certain kinds of practice 
(e.g., games) more enjoyable or less taxing, whereas skilled readers benefit from 
all forms of practice and particularly more integrative formats (e.g., essays). 

For simplicity, and due to overall similar patterns for students who studied 
Body Building or Conclusion Building, data were aggregated across topics for 
this analysis. Pretest and posttest strategy knowledge scores were extracted for 
each students’ studied topic only. 

Exploratory linear models investigated how pretest scores and attitudes, prac- 
tice format, and prior reading ability interacted to predict respective posttest 
scores and attitudes. We examined (a) main effects of pretest, practice format, 
and reading ability; (b) two-way interactions between pretest and format, pretest 
and reading ability, and reading ability and format; and (c) three-way interac- 
tions. For brevity, we summarize these trends here but do not report the full 
models or statistics. 

Overall, across most measures (i.e., topic knowledge, practice interest, prac- 
tice value, and writing interest), we observed no evidence of a main effect of 
practice format—knowledge and motivation gains were not significantly influ- 
enced by practice format alone. However, pretest measures and prior reading 
ability were often significant predictors of posttest measures and demonstrated 
significant two-way interactions. For example, students who possessed more 
strategy knowledge and better reading ability at pretest also demonstrated 
higher strategy knowledge at posttest. Likewise, students who were skilled read- 
ers and who believed (at pretest) that writing practice is more valuable were 
more likely to view writing practice as valuable at posttest. In such cases, two- 
way interactions between pretest measures and reading ability also suggested 
additive effects. Stronger knowledge or more positive attitudes from the outset 
combined productively with reading skill to elicit better outcomes. Three-way 
interactions were difficult to interpret in these models, however. 

To more readily depict and test changes from pretest to posttest, along with 
possible two- and three-way interactions with reading ability, we calculated a 
median split based on students’ prior reading ability, dividing them into “‘less 
skilled readers” and “skilled readers.”” A 2 (study) x 3 (practice format) x 2 
(reading ability) mixed repeated measures ANOVA was then conducted with 
studying as a within-subjects variable (i.e., before studying vs. after studying), 
practice format as a between-subjects variable (1.e., essay-based vs. targeted vs. 
game-based), and reading ability as a between-subjects variable (i.e., less skilled 
vs. skilled). Table 3 reports these data. 

Overall, collapsing across studied topics, students demonstrated significant 
gains in strategy knowledge, F(1,157) = 15.41, p < .001, partial 7° = .089. We also 
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observed an overall increase in interest in practice, F(1,157)=4.64, p=.033, 
partial n?=.029; and a possible increase in perceived value of practice, 
F(1,157) = 3.20, p=.075, partial yn? =.02. General interest in writing decreased 
from pretest to posttest, F(1,157)=5.85, p=.017, partial n°?=.036. Students 
learned about writing strategies from studying and practicing with W-Pal mod- 
ules, and seemed to slightly improve their attitudes toward practice. However, 
they likely experienced some fatigue regarding writing and writing-related 
practice. 

There were no significant two-way interactions between studying and practice 
format for either strategy knowledge or attitudes. In addition, we observed no 
significant two-way interactions between studying and prior reading ability. 
However, we did observe main effects of prior reading ability (i.e., collapsed 
across pretest and posttest). Skilled readers knew more about writing strategies 
than less skilled readers, F(1,157)=23.48, p<.001, partial n= .130, and 
reported a slightly greater interest in writing than less skilled readers, 
F(1,157) = 3.39, p= .068, partial = .021. There was no difference in attitudes 
toward writing practice as a function of prior reading ability. 

Tests of three-way interactions between studying, practice format, and prior 
reading ability revealed several tentative relationships. Most notably, a three- 
way interaction was found for strategy knowledge gains (see Table 3). Skilled 
readers appeared to benefit most from essay-based practice (d= .88), but less 
skilled readers benefited most from game-based practice (d= .62). Three-way 
interactions related to motivation were only marginally significant or nonsigni- 
ficant. Nonetheless, it is worth noting that a fairly consistent pattern emerged 
that warrants future investigation. Skilled readers appeared to find essay-based 
practice and game-based practice more engaging than targeted strategy practice 
without games. That is, skilled readers increased somewhat in their interest and 
value in writing practice in essay and game contexts but exhibited a slight 
decrease in attitudes in strategy-only practice. In contrast, less skilled readers 
appeared to be more engaged by game-based and targeted strategy practice (i.e., 
small gains in practice-related attitudes) than essay-based practice (1.e., slight 
decrement in practice-related attitudes). 

Finally, overall writing interest decreased somewhat for both more and less 
skilled writers. Again, we attribute this effect to fatigue associated with most 
forms of practice and skill acquisition. This decrement seemed to be stronger for 
skilled readers in all contexts, perhaps because they least needed writing instruc- 
tion and practice and thus were the least motivationally engaged by the tasks in 
the study. 


Discussion 


This study considered the relative merits of different forms of practice for writing 
strategy acquisition (RQ1) and writing attitudes (RQ2), and considered whether 
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students’ prior literacy ability led to differential benefits of different practice 
formats (RQ3). Specifically, using the W-Pal system, the study assessed how 
essay-based practice, strategy practice, and game-based strategy practice con- 
tributed to high school adolescents’ learning about writing strategies and atti- 
tudes toward writing and practice. 


Aligning Practice Format, Instructional Goals, and Student 
Characteristics 


Current results corroborated prior research (e.g., Proske et al., 2014) suggesting 
that a variety of practice formats are viable for writing strategy instruction in 
educational technologies. Students gained in strategy knowledge regardless of 
whether they practiced writing essays, completed strategy-based practice tasks, 
or played educational games. Similarly, we observed somewhat positive changes 
in attitudes toward practice across formats. Thus, no singular approach was 
best, but neither did the data offer strong injunctions against any practice 
method. An important implication for educational technology developers and 
educators is that optimal practice design must transcend narrow metrics of 
learning efficacy (i.e., identifying the one “most effective’ format) or engagement 
(i.e., identifying the one “most fun’’ format), despite the appeal of reducing costs 
or streamlining the software. As is the case in traditional writing instruction, 
there are multiple pathways to writing development (Graham et al., 2013; 
Graham & Perin, 2007). 

Logistical constraints such as ease of implementation, cost, and time remain 
nonnegligible; yet, they are not sufficient justification to deny students diverse 
practice opportunities. For example, many AWE developers have already 
invested in robust essay-based practice and formative feedback functions. 
And, within that space, there remain many unanswered questions about how 
to design valid and valued automated writing feedback that will be trusted and 
used by students (Dikli & Bleyle, 2014; Roscoe, Wilson, Johnson, & Mayra, 
2017; Stevenson & Phakiti, 2014). Undoubtedly, there are ample opportunities 
to improve automated supports for writing instruction through innovations that 
provide better feedback. To what extent is it desirable to iteratively improve 
existing tools rather than to “shoehorn in” other practice formats such as edu- 
cational games? Game design is difficult and expensive. 

Indeed, one possible limitation of W-Pal (and the current study) is that its 
development team did not include professional game designers. Games such as 
Fix It, RoBoCo, and LockDown, although functional and instructionally viable, 
might not have met students’ standards for contemporary digital games. 
Although not manipulated in this study, it is plausible that subtle issues of 
game mechanics or aesthetics influence the impact of games on learning. 
Nonetheless, multiple prior studies using W-Pal have reported that students 
do find the games helpful and engaging, and sometimes visually pleasing, despite 
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their lack of professional polish (e.g., Allen et al., 2014; Roscoe et al., 2013; 
Roscoe & McNamara, 2013). And, when games do incorporate more sophisti- 
cated graphics and narratives, the benefits are stronger (e.g., Barab et al., 2012; 
Dickey, 2011). 

Gaming and motivation are also complex, and there is not a simple or 
straightforward connection between game features and student engagement 
(Garris, Ahlers, & Driskell, 2002; Wouters & van Oostendoorp, 2016). In this 
study, we probed attitudes toward practice and writing via straightforward, 
researcher-constructed items (e.g., agreement with statements such as “‘I learn 
valuable things through writing practice’). Results were mixed, showing both 
improvements and decrements in attitudes, and did not differ based on practice 
format. However, students’ feelings toward writing are multidimensional (Troia 
et al., 2012). In addition to interest, prior work has revealed the importance of 
self-efficacy (Bruning, Dempsey, Kauffman, McKim, & Zumbrunn, 2013; 
Pajares, Johnson, & Usher, 2007), epistemic beliefs about writing (Sanders- 
Reio, Alexander, Reio, & Newman, 2014; White & Bruning, 2005), and more. 
Practice format may interact with each (or none) of these factors, and such 
interactions might be mediated or moderated by students’ prior ability or 
other individual differences. Thus, in future work, it will be important to include 
additional motivational constructs and predictors. 

Most importantly, the current study observed interactions between prior 
reading ability and practice format. Skilled readers appeared to gain the most 
from essay-based practice whereas less skilled readers gained most from game- 
based practice, which aligns to prior work on whole- versus part-task training 
methods (Lim et al., 2009; Salden et al., 2006; Wickens et al., 2012, 2013) and 
connections between reading and writing skill development (Fitzgerald & 
Shanahan, 2000; Shanahan, 2015). We argue that essay-based practice allowed 
skilled readers to apply their prior knowledge and newly acquired strategies in 
the context of a complete composition. Skilled readers knew more about writing 
and strategies, and gained more from the lessons and practice activities. This is 
unsurprising given that materials were presented in primarily verbal and textual 
format. Skilled readers were likely better at comprehending the presented infor- 
mation and could link the new information to existing knowledge or mental 
models. As a result, these students were ready to capitalize on and benefit 
from the more complex and integrative nature of whole-task, essay-based prac- 
tice. Skilled readers may have been able to leverage their vocabulary and text 
comprehension abilities to better reflect on the meaning of their new strategies in 
relation to planning, drafting, and revising. 

In contrast, less skilled readers benefitted more from practice that decom- 
posed and targeted specific strategies and skills, and which embedded such 
practice in a game setting. These readers likely lacked a strong repertoire of 
literacy knowledge and skills to draw upon, and the structured format of 
game-based practice provided necessary scaffolding for their strategy 
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acquisition. Less skilled readers may have struggled with simultaneously inter- 
nalizing new strategies and coordinating the full writing process, whereas 
game-based strategy practice better aligned with their needs. For developers 
and educators, such findings imply that technologies need to assess students’ 
existing knowledge and skills (perhaps through stealth assessments conducted as 
students use the system; Shute & Ventura, 2013), and then offer or recommend 
practice activities tailored to those abilities. 

Finally, other aspects of writing development go beyond strategy knowledge 
or enjoyment to encompass communication and interpersonal interactions, 
understanding of audience, and even understanding of oneself. For instance, 
giving, receiving, analyzing, and implementing feedback are essential to writing 
(e.g., Parr & Timperley, 2010; Patchan, Schunn, & Correnti, 2016). Students 
need to learn not only how to compose texts but also how to interact with an 
audience and respond to comments. Iterative essay-based practice with auto- 
mated feedback could nurture these skills by helping students appreciate the 
effects of responding to feedback across essay drafts. Students can reflect on 
how their essay composition was evaluated holistically or with respect to select 
features. Alternatively, games and narratives have been used to explore how 
students view themselves, their roles, and potential careers (i.e., identity forma- 
tion; Barab, Gresalfi, & Ingram-Goble, 2010; Siyahhan, Ingram-Goble, Barab, 
& Solomou, 2017), and games for writing could serve a similar aim (Warren 
et al., 2009). Rather than “writing for the sake of writing,’ games may introduce 
students to new roles and goals that lend an additional sense of purpose. 
Through gaming, for example, students might explore the tasks and responsi- 
bilities of investigative journalists or scientists. In these roles, students might not 
only learn about the importance of verifiable sources, data, clarity, or cohesion, 
but they could also become inspired to pursue writing-based careers. In short, 
the design of activities for computer-supported writing practice may need to 
consider a broader array of instructional goals and alignments with learner, 
educator, and curricular needs. 


Conclusion 


In the classroom, skilled teachers can readily adapt their lessons and practice 
assignments to the needs of individual learners. However, similar levels of flex- 
ibility and personalization can be difficult to achieve in computer-based learning 
environments. Each minor or major variation in potential instructional tasks 
must be built in to the system, often requiring significant time or money. 
Consequently, educational technology developers strive to carefully assess how 
and whether to expend limited resources to create or incorporate different mate- 
rials, practice opportunities, and feedback along with the artificial intelligence 
and natural language algorithms that guide delivery of these features. When an 
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optimal balance is struck, educational technologies can provide powerful tools 
to supplement writing instruction. Our findings suggest that developers, despite 
costs or logistics, should not sidestep the importance of multiple and diverse 
practice opportunities. The ‘‘ideal”’ form of writing practice might not be driven 
by the format itself, but how well the system (like a classroom teacher) can align 
instructional tasks, goals, and student characteristics. Given that most educa- 
tional technologies must serve diverse populations of learners, an important 
implication is that technology developers must aim for flexibility—offering mul- 
tiple forms and types of practice whenever feasible. 


Appendix A: Writing Prompts 


Heroes Prompt 

Having many admirers is one way to become a celebrity, but it is not the way to 
become a hero. Heroes are self-made. Yet in our daily lives we see no difference 
between “‘celebrities” and “‘heroes.”’ For this reason, we deprive ourselves of real 
role models. We should admire heroes—people who are famous because they are 
great, but not celebrities—people who simply seem great because they are famous. 

Should we admire heroes but not celebrities? 
Perfection Prompt 

Too many people do not consider their task or project complete or acceptable 
until every detail has been done right. Fortunately, such people have not always 
had their way, since nothing would ever be completed if we had to check every 
detail before we could consider our work done. In fact, none of the world’s 
greatest accomplishments would have been made, because none of them is per- 
fect in every detail. 

Do people put too much importance on getting every detail right on a project 
or task? 


Appendix B: Strategy Knowledge Questions 


Knowledge assessment questions were presented in a multiple-choice format. 
For brevity, we present the questions below but not all of the answer foil 
options. 


Body Building Assessment Questions 


e What is a characteristic of good body paragraphs? 

e Which of the following best conveys why it is important to support arguments 
with ample evidence? 

e What is one strategy good writers use to increase the persuasiveness of body 
paragraphs? 
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Why is it important to include a body paragraph in an essay? 

Which of the following is a characteristic of strong evidence? 

What is a topic sentence? 

Suppose Mary received the following essay feedback from her teacher: ‘““You 
need to strengthen the arguments in your essay.”” What should Mary do? 
What should writers avoid when writing a body paragraph? 


Conclusion Building Assessment Questions 


What is the most effective strategy for including a thesis in the conclusion 
paragraph? 

What is a property of good conclusion paragraphs? 

Why are conclusion paragraphs important? 

Suppose Bob received the following essay feedback from his teacher: “You 
need to make your conclusion paragraph more meaningful to the reader.” 
What could Bob do? 

Which of the following is a good way to summarize your main ideas in the 
conclusion paragraph? 

What is one strategy good writers use when writing the conclusion of an 
essay? 

Why is it important to include a thesis statement in an essay conclusion? 
What should be avoided when writing a conclusion paragraph? 


Appendix C: Writing Attitudes Survey 


The Writing Attitudes Survey included 12 items, which are grouped below based 
on their common factor. One item was excluded. 


Interest in Strategies and Practice 


Writing practice is exciting. 

Writing practice is entertaining. 

I am excited about writing strategies. 

I like writing strategies. 

Writing practice is so exciting it is easy to pay attention. 
Learning about writing strategies is fascinating. 


Value of Strategies and Practice 


It is important for me to know writing strategies. 
I learn valuable things through writing practice. 
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Interest in Writing 


e | think writing is interesting. 
Writing fascinates me. 
e | am excited about writing. 


Item Excluded from Analyses 


e During writing practice I need something to grab my attention. (reverse 
coded) 
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